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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 

WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )|EI Responsive to communication(s) filed on 19 December 2003 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^3 Claim(s) 1-18 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-18 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



1. 



Claims 1-18 are presented for examination. 



Claim Rejections - 35 USC § 101 



2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claims 7-10 and 18 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. Claims 7 and 10 are directed to machine and/or 
manufacture claims, however, both claims fail to cite any physical article or object and therefore, 
fail to meet the requirements of being a machine or manufacture. 

Claims 8-10 depend on claim 7, and do not overcome the deficiency of claim 7, and 
therefore, are rejected under the same rejection. 



3. The disclosure is objected to because of the following informalities: The abstract cites 
"shared memory are" on line 6, which has a typo, "a shared memory area " should be used 
instead. 

Appropriate correction is required. 



See MPEP 2106 and 2107. 



Specification 



Claim Rejections - 35 USC § 112 



4. 



The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 
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The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claims 1-6 and 12-17 are rejected under 35 U.S.C. 1 12, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Claim 1 recites the limitation 11 said activated modality component 11 in line 4. There is 
insufficient antecedent basis for this limitation in the claim. 

Claims 1 and 2 recite "the modality component" on line 1 and 2, respectively, which is 
not clear as to which modality component it refers to, i.e., each modality component or 
"activated modality component". 

Claims 12 and 13 suffer the same problem in claims 1 and 2 above. 

Double Patenting 

6. Claims 3, 8, 14 are provisionally rejected on the ground of nonstatutory double patenting 
over claims 1, 1 1 and 21 of copending Application No. 10/741,997. This is a provisional double 
patenting rejection since the conflicting claims have not yet been patented. 

The subject matter claimed in the instant application is fully disclosed in the referenced 
copending application and would be covered by any patent granted on that copending application 
since the referenced copending application and the instant application are claiming common 
subject matter, as follows: they are all directed to the same matter even though claim text might 
be different, i.e., registering a plurality of modality components, wherein each of modality 
component handles an interface modality for an application, and for each modality component, 
establishing a list of activation conditions such that at least one operation of the modality 
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component is fired when one of the activation conditions is met, connecting at least one activated 
modality to a device, and conveying user interaction from the device to the modality component 
for processing. 

Furthermore, there is no apparent reason why applicant would be prevented from 
presenting claims corresponding to those of the instant application in the other copending 
application. See In re Schneller, 397 F.2d 350, 158 USPQ 210 (CCPA 1968). See also MPEP 
§ 804. 

Claim Rejections - 35 USC § 102 
1. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

8. Claims 1-4, 6-15 and 17-18 are rejected under 35 U.S.C. 102(e) as being 
unpatentable over Cheyer et al (U.S. 7,069,560 Bl). 

As to claim 1 , Cheyer teaches a method for managing multimodal interactions 
comprising the steps of (col. 5, lines 6-14, col. 10, lines 17-25): 

- registering a plurality of modality components (an agent registers ... vocabulary; col. 7, 
lines 30-34 and col. 17, lines 18-44), wherein each modality component handles an interface 
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modality for an application (a collection ... current inputs; col. 7, lines 13-19 and services that it 
provides, handler, task; col. 12, lines 19-30, lines 46-49, and col. 17, lines 18-44), 

- connecting an activated modality component to a device (user interface agent ... current 
inputs; col. 7, lines 13-19); and 

- conveying a user interaction from the device to the modality component for processing 
(when a facilitator ... facilitator; col. 7, lines 36-43). 

As to claim 2, Cheyer teaches the modality component is registered with a modality 
component server (Facilitator 402 is a specialized server agent; col. 6, lines 31-33 and 45-46 and 
see Figs. 3-4), the method further comprising the step of placing results from the user interaction 
onto a shared memory area of the modality component server (read or write shared data on the 
facilitator; col. 7, lines 47-50). 

As to claim 3, Cheyer teaches for each modality component, establishing a list of 
activation conditions (capabilities, triggers, tasks; col. 17, lines 18-40 and conditions; col. 22, 
lines 21-23) such that at least one operation of the modality component is fired when one of the 
activation conditions is detected (client agent performed the requested service; col. 18, lines 36- 
45 and col. 7, lines 41-43). 

As to claim 4, Cheyer teaches wherein at least one of the plurality of modality 
components is remotely located from the device (a user interface ... sending requests to the 
facilitator ... remote application; col. 8, lines 1-14). 
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As to claim 6, Cheyer teaches wherein at least one of the plurality of modality 
components is disposed within the device (user interface agent runs on the user's local laptop; 
col. 8, lines 2-4). 

As to claim 7, Cheyer teaches a modality component server comprising: 

- a modality activator configured to dynamically activate at least one modality component 
responsive to an occurrence of an application event (a user interface agent; col. 7, lines 13-18 
and col. 8, line 2-7), and 

- a multimodal engine configured to detect an interaction and to responsively initiate an 
interaction response, wherein the interaction and the interaction response have been specified by 
a previously registered modality component (Facilitator agent; col. 16, line 62 - col. 17, line 2 
and col. 18, lines 54-66). 

As to claim 8, Cheyer teaches wherein a plurality of modality components are 
simultaneously utilized, wherein the plurality of modality components specify a plurality of 
interactions and associated interaction responses, and wherein the multimodal engine is 
configured to detect any of the plurality of interactions and to responsively initiate an 
programmatic action (col. 18, lines 54-66 and col. 7, lines 41-48) 

As to claim 9, see rejection of claim 5 above. 
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As to claim 10, Cheyer teaches the multimodal engine is configured to manage 
multimodal interactions involving multiple modality components (col. 7, lines 13-18 and col. 8, 
lines 7-14 and col. 10, lines 16-25). 

As to claim 11, Cheyer teaches the multimodal engine further comprising: 

- a shared memory area (inherent from read and write shared data on the facilitator or 
other agent that maintain shared data; col. 7, lines 45-50), 

- a list of activation conditions (capabilities, triggers, tasks; col. 17, lines 18-40 and 
conditions; col. 22, lines 21-23), and 

- an inference engine configured to run at least one activation condition from the list of 
activation conditions based on a current state of the shared memory area (Facilitator agent; col. 
16, line 62 - col. 17, line 2 and col. 18, lines 54-66). 

As to claim 12, it is the same as the method claim of claim 1 and is rejected under the 
same ground of rejection. 

As to claims 13-15 and 17, see rejections of claims 2-4 and 6 above. 

As to claim 1 8, it is the same as the method claim of claim 1 above and is rejected under 
the same ground of rejection. 
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Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claims 5 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Cheyer et al (U.S. 7,069,560 Bl) in view of Pasternack et al (U.S. 6,859,451 Bl). 

As to claim 5, Cheyer does not explicitly teach wherein the device lacks available 
resources to locally execute at least one function that is handled by the remotely located modality 
component. However, Pasternack teaches the device lacks available resources to locally execute 
at least one function that is handled by the remotely located modality component (col. 2, lines 
53-56). It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to apply the teaching of Pasternack to the system of Cheyer because Pasternack 
provides advantages of having multimodal capability in the server rather only in the user's 
terminal include it enables advanced services to be offered to "thin" clients, and it enables new 
capabilities to be added to services without having to distribute software to user's browser (col. 2, 
lines 49-67). 



As to claim 16, see rejection of claim 5 above. 
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Conclusion 



1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. See PTO 892. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Diem K. Cao whose telephone number is (571) 272-3760. The 
examiner can normally be reached on Monday - Friday, 7:30AM - 3:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, William Thomson can be reached on (571) 272-3718. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



DC 

March 24, 2007 




